N-gram smoothing based on modeling of expectation of n-gram occurrence

Discriminative n-gram language modeling

This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on maximizing the regularized conditional log-likelihood. The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are t...

n-Gram Geo-trace Modeling

As location-sensing smart phones and location-based services gain mainstream popularity, there is increased interest in developing techniques that can detect anomalous activities. Anomaly detection capabilities can be used in theft detection, remote elder-care monitoring systems, and many other applications. In this paper we present an ngram based model for modeling a user’s mobility patterns. ...

Modified Kneser-Ney Smoothing of n-gram Models

n-Gram-Based Text Compression

We propose an efficient method for compressing Vietnamese text using n-gram dictionaries. It has a significant compression ratio in comparison with those of state-of-the-art methods on the same dataset. Given a text, first, the proposed method splits it into n-grams and then encodes them based on n-gram dictionaries. In the encoding phase, we use a sliding window with a size that ranges from bi...

N-gram-based Text Attribution

Quantitative authorship attribution refers to the task of identifying the author of a text based on measurable features of the author’s style—a problem that has practical application in areas as diverse as literary scholarship, plagiarism detection, and criminal forensics. Attribution methods generally follow a generative approach, wherein a statistical “profile” is created for a set of candida...